Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Introduce cache for speeding up APIServices lookup #79999

Merged
merged 1 commit into from
Aug 13, 2019
Merged

Introduce cache for speeding up APIServices lookup #79999

merged 1 commit into from
Aug 13, 2019

Conversation

tedyu
Copy link
Contributor

@tedyu tedyu commented Jul 10, 2019

What type of PR is this?
/kind cleanup

What this PR does / why we need it:
This PR addresses the TODO item above AvailableConditionController#addService

A cache is introduced into AvailableConditionController. In getAPIServicesFor, we check whether the v1.Service reference has been queried before.
If so, return the cached result directly.

NONE

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jul 10, 2019
@tedyu
Copy link
Contributor Author

tedyu commented Jul 10, 2019

/assign @deads2k

@k8s-ci-robot k8s-ci-robot added sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jul 10, 2019
@roycaihw
Copy link
Member

/cc @liggitt

@liggitt
Copy link
Member

liggitt commented Jul 11, 2019

given the volume of endpoint updates, I think optimizing getAPIServicesFor is probably worthwhile, but I would recommend something more like #79999 (comment)

this will also need thorough tests to prove that APIService additions/updates/removals result in the proper map of service-namespace/service-name -> apiservice name list

@liggitt
Copy link
Member

liggitt commented Jul 11, 2019

and since this is being done in service of performance, quantifying the gain with a unit test benchmark is recommended

@liggitt
Copy link
Member

liggitt commented Jul 11, 2019

/hold
for rework

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 11, 2019
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jul 11, 2019
@tedyu
Copy link
Contributor Author

tedyu commented Jul 11, 2019

@liggitt
I tried to address your comments.
Please take a look.

I will work on benchmark next.

@tedyu tedyu force-pushed the avail-ctrl-cache branch 2 times, most recently from ceb7366 to 66b0c27 Compare July 12, 2019 00:06
@tedyu
Copy link
Contributor Author

tedyu commented Jul 12, 2019

For benchmark test, I was looking at endpoints_controller_test.go
But it seems XXEndpoints funcs are not exercised by the test.

@tedyu
Copy link
Contributor Author

tedyu commented Jul 18, 2019

@deads2k
Please provide your feedback.

@tedyu
Copy link
Contributor Author

tedyu commented Jul 18, 2019

@liggitt
Can you advise how to proceed ?

thanks

@liggitt
Copy link
Member

liggitt commented Jul 18, 2019

This looks reasonable to me. @deads2k can take a look early next week and it should be good to merge after that

@tedyu
Copy link
Contributor Author

tedyu commented Jul 22, 2019

@deads2k
Please review the PR.

thanks

@tedyu
Copy link
Contributor Author

tedyu commented Jul 23, 2019

@deads2k
gentle ping for review.

@tedyu
Copy link
Contributor Author

tedyu commented Jul 25, 2019

@deads2k @liggitt
I have added the comment above rebuildAPIServiceCache func.

Please take another look.

Thanks

@tedyu
Copy link
Contributor Author

tedyu commented Jul 26, 2019

@deads2k @liggitt
There was bug in my previous change where if castObj or oldCastObj is nil, there would still be panic.

This has been fixed in the latest PR.

Please take a look.

@tedyu
Copy link
Contributor Author

tedyu commented Aug 5, 2019

@deads2k
Please review this PR.

thanks

@tedyu
Copy link
Contributor Author

tedyu commented Aug 8, 2019

@deads2k
If you have high level question / suggestion, I would be happy to address them.

@tedyu
Copy link
Contributor Author

tedyu commented Aug 13, 2019

@liggitt
Your comment has been addressed.

@liggitt
Copy link
Member

liggitt commented Aug 13, 2019

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 13, 2019
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: liggitt, tedyu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 13, 2019
@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

1 similar comment
@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@k8s-ci-robot k8s-ci-robot merged commit ea59354 into kubernetes:master Aug 13, 2019
@k8s-ci-robot k8s-ci-robot added this to the v1.16 milestone Aug 13, 2019
@k8s-ci-robot
Copy link
Contributor

@tedyu: The following tests failed, say /retest to rerun them all:

Test name Commit Details Rerun command
pull-kubernetes-e2e-gce 7db2c85 link /test pull-kubernetes-e2e-gce
pull-kubernetes-e2e-gce-100-performance 7db2c85 link /test pull-kubernetes-e2e-gce-100-performance

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@liggitt liggitt liggitt left review comments

@deads2k deads2k deads2k left review comments

@yue9944882 yue9944882 yue9944882 left review comments

@caesarxuchao caesarxuchao Awaiting requested review from caesarxuchao

@sttts sttts Awaiting requested review from sttts

Assignees

@liggitt liggitt

@deads2k deads2k

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note-none Denotes a PR that doesn't merit a release note. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/scalability Categorizes an issue or PR as relevant to SIG Scalability. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
v1.16
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@tedyu @roycaihw @liggitt @k8s-ci-robot @fejta-bot @deads2k @yue9944882